High-radix Interconnection Networks a Dissertation Submitted to the Department of Electrical Engineering and the Committee on Graduate Studies of Stanford University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

Over the past twenty years, the pin bandwidth available to a chip has increased by approximately an order of magnitude every five years – a rate very similar to Moore’s Law. This increasing pin bandwidth can be effectively utilized by creating high-radix routers with large number of skinny ports instead of low-radix routers with fat ports. The use of high-radix routers leads to lower cost and better performance, but highradix networks present many difficult challenges. This thesis explores challenges in scaling to high-radix routers, including topology, routing, and router microarchitecture. Topology is a critical aspect of any interconnection network as it sets performance bounds and determines the cost of the network. This thesis presents a cost-efficient high-radix topology referred to as the flattened butterfly topology which exploits the availability of high-radix routers. Compared to a folded-Clos topology, the flattened butterfly provides approximately 2× reduction in cost per performance on balanced traffic while maintaining the same cost per performance on adversarial traffic pattern. Given the topology, routing determines the path between the source and its destination. Proper routing is required to exploit the path diversity available in a high-radix network and we discuss the advantages of using adaptive routing in highradix networks as well as how non-minimal routing is critical to properly exploiting the flattened butterfly topology. Conventional microarchitectures do not scale to high radix since the complexity of the allocators in the routers scale quadratically with the radix. This thesis presents a hierarchical router organization that results in a distributed, complexity-effective